Security application using camera SOC with multi-sensor capabilities

ABSTRACT

An apparatus includes a first lens and first image sensor, a second lens and second image sensor, a first motion sensor, a second motion sensor, and a processor. The first image sensor may be configured to capture a first video image stream of a first field of view (FOV). The second image sensor may be configured to capture a second video image stream of a second FOV. The first motion sensor may be configured to detect motion in the first FOV. The second motion sensor may be configured to detect motion in the second FOV. The processor is generally coupled to the first image sensor, the first motion sensor, the second image sensor, and the second motion sensor, and configured to generate a third video image stream in response to one or more of the first video image stream and the second video image stream.

FIELD OF THE INVENTION

The invention relates to security cameras generally and, moreparticularly, to a method and/or apparatus for implementing a securityapplication using a camera system on chip (SOC) with multi-sensorcapabilities.

BACKGROUND

Home security systems often utilize two cameras to watch cornerlocations such as a driveway and a side-yard, or a driveway and afront-door pathway. The two cameras record the two areas separately (intwo video files). When someone walks from the driveway to the side-yardor the front door, tracking the movement requires switching between thetwo video files. For consumers, purchasing and installing twoindependent cameras to cover such locations (i.e., corners), in order towatch for activities from two directions, is expensive and tedious.Currently, some camera manufacturers have been looking for ways to havetwo cameras inter-connected and predict what will happen so that themain camera activates the secondary camera ahead of time. Besides thecost of the second camera, it can be difficult to install and get thetwo independent cameras to interact accurately.

It would be desirable to implement a security application using a camerasystem on chip (SOC) with multi-sensor capabilities.

SUMMARY

The invention concerns an apparatus including a first lens and firstimage sensor, a second lens and second image sensor, a first motionsensor, a second motion sensor, and a processor. The first lens andfirst image sensor may be configured to capture a first video imagestream of a first field of view (FOV). The second lens and second imagesensor may be configured to capture a second video image stream of asecond FOV. The first motion sensor may be configured to detect motionin the first field of view (FOV). The second motion sensor may beconfigured to detect motion in the second field of view (FOV). Theprocessor is generally coupled to the first image sensor, the firstmotion sensor, the second image sensor, and the second motion sensor,and configured to generate a third video image stream in response to oneor more of the first video image stream and the second video imagestream.

BRIEF DESCRIPTION OF THE FIGURES

Embodiments of the invention will be apparent from the followingdetailed description and the appended claims and drawings in which:

FIG. 1 is a diagram illustrating a context of an example embodiment ofthe invention.

FIG. 2 is a diagram illustrating fields of vision of a camera inaccordance with an embodiment of the invention.

FIG. 3 is a diagram illustrating an example implementation in accordancewith an embodiment of the invention.

FIG. 4 is a diagram illustrating components of an example implementationin accordance with an embodiment of the invention.

FIG. 5 is a diagram of an example processing circuit.

FIG. 6 is a diagram illustrating a process in accordance with an exampleembodiment of the invention.

FIGS. 7A-7B are a diagram illustrating another example process inaccordance with an example embodiment of the invention.

DETAILED DESCRIPTION OF THE EMBODIMENTS

Embodiments of the present invention include providing a securityapplication using a camera system on chip (SOC) with multi-sensorcapabilities that may (i) be used in corner surveillance applications,(ii) reduce cost, (iii) connect multiple sensors to a single camera SoC,(iv) provide low power (e.g., battery) operation, (v) automaticallyswitch sensors on and off to maintain low power operation, (vi) be usedin residential settings, (vii) reduce storage (memory) needs/costs,(viii) provide ease of installation, (ix) utilize video analytics topredict target motion, (x) provide seamless tracking from one field ofview to another, and/or (xi) be implemented as one or more integratedcircuits.

In various embodiments, multi-sensor capabilities of a camera system onchip (SoC) may be utilized to build a battery-powered camera thatsupports multiple sensors. The multi-sensor camera may be configured toprovide surveillance in a corner configuration. In the followingdescription, an example of a camera utilizing two sensors is describedfor clarity. However, it will be apparent to those skilled in the fieldof the invention that the number of sensors may easily be extended tomore than two sensors.

A corner configuration is generally used where a field of view (FOV) tobe covered is greater than about 180 degrees (e.g., 270 degrees, etc.),but less than 360 degrees due to an obstacle (e.g., building wall,etc.). In various embodiments, standard configurations may be madeavailable to fit various residential configurations. In an embodimentwith two sensors, both sensors may be connected to a single camerasystem on chip (SoC) for low power (e.g., battery) operation. In anexample embodiment configured to cover two fields of view (FOVs), afirst image sensor and a first motion sensor (e.g., passive infrared(PIR) sensor) may be directed in a first direction and a second imagesensor and a second motion sensor (e.g., passive infrared (PIR) sensor)may be directed in a second direction, where the second direction is atan angle (e.g., orthogonal) to the first direction. The passive infrared(PIR) sensors generally use very little power.

In an example operation, the first passive infrared (PIR) sensor and thesecond passive infrared (PIR) sensor may be in an activated state andthe first image sensor and the second image sensor may be in anon-activated stated. When someone walks to the driveway, the firstpassive infrared (PIR) sensor may be triggered. In response to the firstPIR sensor being triggered, the camera SoC may turn on the first imagesensor and start generating a video stream comprising video capturedfrom the first image sensor. The second image sensor may be left in thenon-activated state during this time. When the person walks around thecorner, the second passive infrared sensor paired with the second imagesensor may be triggered. In response to the second PIR sensor beingtriggered, the camera SoC may turn on the second image sensor, turn offthe first image sensor, and continue generating the video stream usingvideo captured from the second image sensor. In some embodiments, thecamera SoC may be configured to blend (or stitch) the video from the twoimage sensors to provide a smooth (seamless) transition between imagesof the two cameras. In various embodiments, the video stream generatedby the camera SoC may be stored for later playback.

A system in accordance with embodiments of the invention generallyprovides multiple benefits. A camera user (or manufacturer) may realizereduced cost (e.g., instead of two cameras, one camera and a secondsensor and lens may be purchased and installed). Consumers may enjoyeasier installation, and lower cost for such installation. A singlevideo recording (file) may be created instead of two video files,lowering storage costs. The recorded video may be naturally “seamless”(e.g., recording movement towards the driveway and then around thecorner, etc.). Video analytics may also be run on the camera SoC topredict the direction of the movement and/or reduce false detections.Predicting the direction of motion generally allows the second imagesensor to be started ahead of time, to be ready as soon as the movingobject of interest is in the field of view of the second image sensor.The video analytics may allow extended battery time by minimizing theamount of time the image sensors are actually active.

Referring to FIG. 1, a diagram is shown illustrating a context in whichan example embodiment of the invention may be implemented. In anexample, a residential setting may include a house 90. The house 90 maypresent a number of corner locations. In an example, a camera 100 may beplaced at a corner location between a side of the house 90 facing adriveway area and a side of the house 90 facing a side yard or frontdoor pathway. In an example, the camera 100 may be mounted to a soffitof the house 90. In another example, the camera 100 may be mounted tothe two walls of the house 90 (e.g., using an angle brackets). Thecamera 100 may be directed toward an environment adjacent to the sidesof the house 90 encompassing the corner location of the camera 100. Inan example, the camera 100 may be a battery-powered camera.

In an example embodiment configured to cover two fields of view (FOVs),a first image sensor and a first passive infrared (PIR) sensor may bedirected in a first direction and a second image sensor and a secondpassive infrared (PIR) sensor may be directed in a second direction,where the second direction is at an angle (e.g., orthogonal) to thefirst direction. The passive infrared (PIR) sensors generally use verylittle power.

In an example, the camera 100 may be configured to cover two fields ofview (FOVs). A first field of view (FOV) may encompass the areaincluding the driveway. A second field of view (FOV) may encompass thearea including the side-yard or front-door pathway. In an example, thecamera 100 may comprise a first image sensor, a first passive infraredsensor, a second image sensor, a second passive infrared sensor and acamera system on chip (SoC). In an example, the first image sensor andthe first passive infrared (PIR) sensor may be directed toward the firstfield of view and the second image sensor and the second passiveinfrared sensor may be directed toward the second field of view. Thepassive infrared (PIR) sensors generally use very little power.

In an example operation, the first passive infrared (PIR) sensor and thesecond passive infrared (PIR) sensor may be in the activated state andthe first image sensor and the second image sensor may be in anon-activated stated. When an object moves to the driveway, the firstpassive infrared (PIR) sensor may be triggered. In response to the firstPIR sensor being triggered, the camera SoC may turn on the first imagesensor and start generating a video stream comprising video capturedfrom first image sensor. The second image sensor may be left in thenon-activated state during this time. When the object moves around thecorner (e.g., towards the front door), the second passive infraredsensor paired with the second image sensor may be triggered. In responseto the second PIR sensor being triggered, the camera SoC may turn on thesecond image sensor, turn off the first image sensor, and continuegenerating the video stream using video captured from the second imagesensor. The camera SoC may be configured to provide a seamlesstransition between the video captured from the two images sensors.

Referring to FIG. 2, a diagram is shown illustrating example fields ofview (FOVs) of the camera 100 of FIG. 1. In an example, the camera 100is generally configured to have a first viewing angle 102 for the firstfield of view and a second viewing angle 104 for the second field ofview. The viewing angles 102 and 104 may be wide viewing angles (e.g.,less than or substantially equal to 180 degrees). In an example, the twoviewing angles 102 and 104 may overlap (e.g., by one or more degrees) atthe corner location of the house 90. In an example, the camera 100 mayutilize two fisheye lenses to provide the two viewing angles 102 and104. In various embodiments, the camera 100 may be connected to (or bepart of) a home security system.

In an example, the camera 100 may comprise processing circuitry (e.g.,the camera SoC) configured to perform a de-warping operation to provideviews of particular portions (e.g., right, center, left, etc.) of thetwo viewing angles 102 and 104. The de-warping operation generallyrefers to a process of correcting a perspective of an image to reverseeffects of geometric distortions (e.g., caused by a camera lens).De-warping may allow the camera 100 to cover the wide viewing angles 102and 104 (e.g., using fisheye or panoramic lenses), while still having a“normal” view of an otherwise distorted or reversed image. De-warpingmay also allow the camera 100 to seamlessly combine images captured fromthe two viewing angles into a single video stream.

Referring to FIG. 3, a diagram of the camera 100 is shown illustratingan example implementation in accordance with an example embodiment ofthe invention. In an example, the camera 100 may comprise a housing 106,a number of blocks (or circuits) 108 a-108 n, and/or a block (orcircuit) 110. The housing 106 may comprise an upper portion 112 and alower portion 114. The upper portion 112 may be configured to mount thecamera 100 to a structure (e.g., a soffit or wall of the house 90). Inan example, the lower portion 114 may be implemented as a transparentdome. The housing 106 is generally configured to protect components ofthe camera 100 from the environment and tampering. The blocks 108 a-108n may comprise lens and sensor assemblies. In FIG. 3, a first lens andsensor assembly 108 a and a second lens and sensor assembly 108 n areshown. The block 110 may comprise a processor (or system-on-chip (SoC)).In various embodiments, the block 110 may implement a camera SoC withmulti-sensor capabilities.

In various embodiments, each of the lens and sensor assemblies 108 a-108n may have a respective field of view (or viewing angle). In an example,the respective viewing angles of the lens and sensor assemblies 108a-108 n may be combined (with or without overlap) to provide a desirednumber of degrees of coverage for a variety of corner configurations. Inan example, two lens and sensor assemblies 108 a and 108 n may beconfigured to provide coverage for a 270 degrees field of view (asdescribed above in connection with FIG. 2). For example, the lens andsensor assembly 108 a may be configured to observe the viewing angle 102and the lens and sensor assembly 108 n may be configured to observe theviewing angle 104.

The lens and sensor assemblies 108 a and 108 n may be configured todetect and/or measure various types of input from the environment (e.g.,light, motion, heat, sound, smoke, carbon monoxide, Wi-Fi signals,etc.). In an example, each of the lens and sensor assemblies 108 a and108 n may comprise a lens assembly, an image sensor, and a motion sensor(described below in connection with FIG. 4). In an example, the motionsensor may be implemented as a passive infrared (PIR) sensor. In anotherexample, the motion sensor may be a smart motion sensor based on vision.In another example, the lens and sensor assemblies 108 a and 108 n mayfurther comprise a microphone configured to measure audio levels. Inanother example, a directional microphone may be implemented to allow alocation of a noise source to be determined. Other blocks (or circuitsor components) of the camera 100 may be implemented. The components ofthe camera 100 may be varied according to the design criteria of aparticular implementation.

The lens and sensor assemblies 108 a and 108 n may be configured tocapture video of respective fields of view. The edges of the field ofview of the lens and sensor assembly 108 a may be illustrated by thelong-dashed lines of the viewing angle 102 in FIG. 2. The edges of thefield of view of the lens and sensor assembly 108 n may be illustratedby the short-dashed lines of the viewing angle 104 in FIG. 2. In anexample, the fields of view of the lens and sensor assemblies 108 a and108 n may overlap. The range of the fields of view provided by theviewing angles 102 and 104 may be varied according to the designcriteria of a particular implementation.

Each of the lens and sensor assemblies 108 a and 108 n may be directedtowards a location in the overall field of view of the camera 100. Eachof the lens and sensor assemblies 108 a and 108 n may provide coveragefor a portion of the field of view of the camera 100. In an example, thelens and sensor assembly 108 a may provide coverage for the viewingangle 102. In another example, the lens and sensor assembly 108 n mayprovide coverage for the viewing angle 104. The portion of coverage ofeach of the lens and sensor assemblies 108 a and 108 n may be a zone. Inan example, a first zone may cover the viewing angle 102 and be coveredby the lens and sensor assembly 108 a. In another example, a second zonemay cover the viewing angle 104 and be covered by the lens and sensorassembly 108 n. In yet another example, additional zones may coverportions of the field of view of the camera 100 and be covered byrespective ones of a number of lens and sensor assemblies 108 a-108 n.While the viewing angles 102 and 104 are shown overlapping, in someembodiments the zones covered by the lens and sensor assemblies 108a-108 n may be configured so as to not overlap. The number, size and/orarrangement of the zones may be varied according to the design criteriaof a particular implementation.

Referring to FIG. 4, a diagram is shown illustrating an exampleimplementation of a camera in accordance with an example embodiment ofthe invention. In an example, each of the lens and sensor assemblies 108a-108 n may comprise a lens assembly 60 a-60 n, a motion sensor 70 a-70n, and an image sensor 80 a-80 n (not shown). The lens and sensorassemblies 108 a-108 n may be arranged such that optical axes of thelens assemblies 60 a-60 n are at an angle to one another. In an exampleimplementing two lens and sensor assemblies, the optical axes may be atan angle of 90 degrees to one another.

In various embodiments, each of the lens and sensor assemblies 108 a-108n is generally connected to a single processor or system on chip (SoC)110 by one or more buses. In an example, the lens and sensor assemblies108 a-108 n may be connected to the processor or system on chip (SoC)110 using one or more serial buses (e.g., I²C, SPI, etc.), parallelbuses (e.g. GPIO, etc.), and/or individual signals (e.g., via wires ortraces). In various embodiments, the lens and sensor assemblies 108a-108 n may communicate video image streams and motion detection signalsto the processor or system on chip (SoC) 110, and the processor orsystem on chip (SoC) 110 may communicate control signals to the lens andsensor assemblies 108 a-108 n.

Referring to FIG. 5, a block diagram of the camera 100 is shownillustrating a camera system-on-a-chip connected to multiple lens andsensor assemblies 108 a-108 n. The camera 100 may comprise the lenses 60a-60 n, the motion sensors 70 a-70 n, the image sensors 80 a-80 n, theSoC 110, a block (or circuit) 112, a block (or circuit) 114, and/or ablock (or circuit) 116. The circuit 112 may be implemented as a memory.The block 114 may be a communication module. The block 116 may beimplemented as a battery. In some embodiments, the camera 100 maycomprise the lenses 60 a-60 n, the motion sensors 70 a-70 n, the imagesensors 80 a-80 n, the SoC 110, the memory 112, the communication module114, and the battery 116. In another example, the camera 100 maycomprise the lenses 60 a-60 n, the motion sensors 70 a-70 n, and thecapture devices 80 a-80 n, and the SoC 110, the memory 112, thecommunication module 114, and the battery 116 may be components of aseparate device. The implementation of the camera 100 may be variedaccording to the design criteria of a particular implementation.

The lenses 60 a-60 n are shown attached to respective capture devices 80a-80 n. In an example, the capture devices 80 a-80 n are shownrespectively comprising blocks (or circuits) 82 a-82 n, blocks (orcircuits) 84 a-84 n and blocks (or circuits) 86 a-86 n. The circuits 82a-82 n may be sensors (e.g., image sensors). The circuits 84 a-84 n maybe processors and/or logic. The circuits 86 a-86 n may be memorycircuits (e.g., frame buffers).

The capture devices 80 a-80 n may be configured to capture video imagedata (e.g., light collected and focused by the lenses 60 a-60 n). Thecapture devices 80 a-80 n may capture data received through the lenses60 a-60 n to generate a video bitstream (e.g., a sequence of videoframes). The lenses 60 a-60 n may be directed, tilted, panned, zoomedand/or rotated to capture the environment surrounding the camera 100(e.g., capture data from the fields of view).

The capture devices 80 a-80 n may transform the received light into adigital data stream. In some embodiments, the capture devices 80 a-80 nmay perform an analog to digital conversion. For example, the capturedevices 80 a-80 n may perform a photoelectric conversion of the lightreceived by the lenses 60 a-60 n. The image sensors 80-80 n maytransform the digital data stream into a video data stream (orbitstream), a video file, and/or a number of video frames. In anexample, each of the capture devices 80 a-80 n may present the videodata as a digital video signal (e.g., the signals VIDEO_A-VIDEO_N). Thedigital video signals may comprise the video frames (e.g., sequentialdigital images and/or audio).

The video data captured by the capture devices 80 a-80 n may berepresented as signals/bitstreams/data VIDEO_A-VIDEO_N (e.g., a digitalvideo signal). The capture devices 80 a-80 n may present the signalsVIDEO_A-VIDEO_N to the processor/SoC 110. The signals VIDEO_A-VIDEO_Nmay represent the video frames/video data. The signals VIDEO_A-VIDEO_Nmay be video streams captured by the capture devices 80 a-80 n.

The image sensors 82 a-82 n may receive light from the respective lenses60 a-60 n and transform the light into digital data (e.g., thebitstream). For example, the image sensors 82 a-82 n may perform aphotoelectric conversion of the light from the lenses 60 a-60 n. In someembodiments, the image sensors 82 a-82 n may have extra margins that arenot used as part of the image output. In some embodiments, the imagesensors 82 a-82 n may not have extra margins. In some embodiments, someof the image sensors 82 a-82 n may have the extra margins and some ofthe image sensors 82 a-82 n may not have the extra margins. In someembodiments, the image sensors 82 a-82 n may be configured to generatemonochrome (B/W) video signals. In some embodiments, the image sensors82 a-82 n may be configured to generate color (e.g., RGB, YUV, RGB-IR,YCbCr, etc.) video signals. In some embodiments, the image sensors 82a-82 n may be configured to generate video signals in response tovisible and/or infrared (IR) light.

The processor/logic 84 a-84 n may transform the bitstream into a humanviewable content (e.g., video data that may be understandable to anaverage person regardless of image quality, such as the video frames).For example, the processors 84 a-84 n may receive pure (e.g., raw) datafrom the camera sensors 82 a-82 n and generate (e.g., encode) video data(e.g., the bitstream) based on the raw data. The capture devices 80 a-80n may have the memory 86 a-86 n to store the raw data and/or theprocessed bitstream. For example, the capture devices 80 a-80 n mayimplement the frame memory and/or buffers 86 a-86 n to store (e.g.,provide temporary storage and/or cache) one or more of the video frames(e.g., the digital video signal). In some embodiments, theprocessors/logic 84 a-84 n may perform analysis and/or correction on thevideo frames stored in the memory/buffers 86 a-86 n of the capturedevices 80 a-80 n.

The motion sensors 70 a-70 n may be configured to detect motion (e.g.,in the fields of view corresponding to the viewing angles 102 and 104).The detection of motion may be used as one threshold for activating thecapture devices 80 a-80 n. The motion sensors 70 a-70 n may beimplemented as internal components of the camera 100 and/or ascomponents external to the camera 100. In an example, the sensors 70a-70 n may be implemented as passive infrared (PIR) sensors. In anotherexample, the sensors 70 a-70 n may be implemented as smart motionsensors. In an example, the smart motion sensors may comprise lowresolution image sensors configured to detect motion and/or persons. Themotion sensors 70 a-70 n may each generate a respective signal (e.g.,SENS_A-SENS_N) in response to motion being detected in one of therespective zones (e.g., FOVs 102 and 104). The signals SENS_A-SENS_N maybe presented to the processor/SoC 110. In an example, the motion sensor70 a may generate (assert) the signal SENS_A when motion is detected inthe FOV 102 and the motion sensor 70 n may generate (assert) the signalSENS_N when motion is detected in the FOV 104.

The processor/SoC 110 may be configured to execute computer readablecode and/or process information. The processor/SoC 110 may be configuredto receive input and/or present output to the memory 112. Theprocessor/SoC 110 may be configured to present and/or receive othersignals (not shown). The number and/or types of inputs and/or outputs ofthe processor/SoC 110 may be varied according to the design criteria ofa particular implementation. The processor/SoC 110 may be configured forlow power (e.g., battery) operation.

The processor/SoC 110 may receive the signals VIDEO_A-VIDEO_N and thesignals SENS_A-SENS_N. The processor/SoC 110 may generate a signal METAbased on the signals VIDEO_A-VIDEO_N, the signals SENS_A-SENS_N, and/orother input. In some embodiments, the signal META may be generated basedon analysis of the signals VIDEO_A-VIDEO_N and/or objects detected inthe signals VIDEO_A-VIDEO_N. In various embodiments, the processor/SoC110 may be configured to perform one or more of feature extraction,object detection, object tracking, and object identification. Forexample, the processor/SoC 110 may determine motion information byanalyzing a frame from the signals VIDEO_A-VIDEO_N and comparing theframe to a previous frame. The comparison may be used to perform digitalmotion estimation.

In some embodiments, the processor/SoC 110 may perform video stitchingoperations. The video stitching operations may be configured tofacilitate seamless tracking as objects move through the fields of viewassociated with the capture devices 80 a-80 n. The processor/SoC 110 maygenerate a number of signals VIDOUT_A-VIDOUT_N. The signalsVIDOUT_A-VIDOUT_N may be portions (components) of a multi-sensor videosignal. In some embodiments, the processor/SoC 110 may be configured togenerate a single video output signal (e.g., VIDOUT). The video outputsignal(s) (e.g., VIDOUT or VIDOUT_A-VIDOUT_N) may be generatedcomprising video data from one or more of the signals VIDEO_A-VIDEO_N.The video output signal(s) (e.g., VIDOUT or VIDOUT_A-VIDOUT_N) may bepresented to the memory 112 and/or the communications module 114.

The memory 112 may store data. The memory 112 may be implemented as acache, flash memory, memory card, DRAM memory, etc. The type and/or sizeof the memory 112 may be varied according to the design criteria of aparticular implementation. The data stored in the memory 112 maycorrespond to a video file, motion information (e.g., readings from thesensors 70 a-70 n, video stitching parameters, image stabilizationparameters, user inputs, etc.) and/or metadata information.

The lenses 60 a-60 n (e.g., camera lenses) may be directed to provide aview of an environment surrounding the camera 100. The lenses 60 a-60 nmay be aimed to capture environmental data (e.g., light). The lenses 60a-60 n may be wide-angle lenses and/or fish-eye lenses (e.g., lensescapable of capturing a wide field of view). The lenses 60 a-60 n may beconfigured to capture and/or focus the light for the capture devices 80a-80 n. Generally, the image sensors 82 a-82 n are located behind thelenses 60 a-60 n. Based on the captured light from the lenses 60 a-60 n,the capture devices 80 a-80 n may generate bitstreams and/or video data.

The communications module 114 may be configured to implement one or morecommunications protocols. For example, the communications module 114 maybe configured to implement Wi-Fi, Bluetooth, Ethernet, etc. Inembodiments where the camera 100 is implemented as a wireless camera,the protocol implemented by the communications module 114 may be awireless communications protocol. The type of communications protocolsimplemented by the communications module 114 may be varied according tothe design criteria of a particular implementation.

The communications module 114 may be configured to generate a broadcastsignal as an output from the camera 100. The broadcast signal may sendthe video data VIDOUT to external devices. For example, the broadcastsignal may be sent to a cloud storage service (e.g., a storage servicecapable of scaling on demand). In some embodiments, the communicationsmodule 114 may not transmit data until the processor/SoC 110 hasperformed video analytics to determine that an object is in the field ofview of the camera 100.

In some embodiments, the communications module 114 may be configured togenerate the manual control signal. The manual control signal may begenerated in response to a signal from a user received by thecommunications module 114. The manual control signal may be configuredto activate the processor/SoC 110. The processor/SoC 110 may beactivated in response to the manual control signal regardless of thepower state of the camera 100.

The camera 100 may include a battery 116 configured to provide power forthe various components of the camera 100. The multi-step approach toactivating and/or disabling the capture devices 80 a-80 n based on theoutputs of the motion sensors 70 a-70 n and/or any other power consumingfeatures of the camera 100 may be implemented to reduce a powerconsumption of the camera 100 and extend an operational lifetime of thebattery 116. The motion sensors 70 a-70 n may have a very low drain onthe battery 116 (e.g., less than 10 W). In an example, the motionsensors 70 a-70 n may be configured to remain on (e.g., always active)unless disabled in response to feedback from the processor/SoC 110. Thevideo analytics performed by the processor/SoC 110 may have a largedrain on the battery 116 (e.g., greater than the motion sensors 70 a-70n). In an example, the processor/SoC 110 may be in a low-power state (orpower-down) until some motion is detected by the motion sensors 70 a-70b.

The camera 100 may be configured to operate using various power states.For example, in the power-down state (e.g., a sleep state, a low-powerstate) the motion sensors 70 a-70 n and the processor/SoC 110 may be onand other components of the camera 100 (e.g., the image capture devices80 a-80 n, the memory 112, the communications module 114, etc.) may beoff. In another example, the camera 100 may operate in an intermediatestate. In the intermediate state, one of the image capture devices 80a-80 n may be on and the memory 112 and/or the communications module 114may be off. In yet another example, the camera 100 may operate in apower-on (or high power) state. In the power-on state, the motionsensors 70 a-70 n, the processor/SoC 110, the capture devices 80 a-80 n,the memory 112 and/or the communications module 114 may be on. Thecamera 100 may consume some power from the battery 116 in the power-downstate (e.g., a relatively small and/or minimal amount of power). Thecamera 100 may consume more power from the battery 116 in the power-onstate. The number of power states and/or the components of the camera100 that are on while the camera 100 operates in each of the powerstates may be varied according to the design criteria of a particularimplementation.

Referring to FIG. 6, a diagram is shown illustrating a process inaccordance with an example embodiment of the invention. In an example, amethod (or process) 200 may be performed using the camera 100. Themethod 200 may detect motion within a monitored area and provide a videorecord of the object in motion detected. The method 200 generallycomprises a decision step (or state) 202, a decision step (or state)204, a step (or state) 206, and a step (or state) 208.

The process 200 may start in either the decision state 202 or thedecision state 204. In the decision state 202, the processor 110 maydetermine whether a first motion sensor (e.g., PIR-1) has beentriggered. If the first motion sensor PIR-1 has not been triggered, theprocess 200 may move to the decision state 204. If the first motionsensor PIR-1 has been triggered, the process 200 may move to the state206. In the decision state 204, the processor 110 may determine whethera second motion sensor (e.g., PIR-2) has been triggered. If the secondmotion sensor PIR-2 has not been triggered, the process 200 may move tothe decision state 202. If the second motion sensor PIR-2 has beentriggered, the process 200 may move to the state 208. The process 200may loop through the decision states 202 and 204 until either the firstor the second motion sensor is triggered.

In the state 206, the processor 110 may activate a first image sensor(e.g., CAMERA 1) corresponding to the first motion sensor and recordvideo. If the camera 100 is in a low power mode, the processor 110 maydetermine whether a second camera (e.g., CAMERA 2) associated with thesecond motion sensor is on and, if so, deactivate the second camera.

In the state 208, the processor 110 may activate the second image sensor(e.g., CAMERA 2) corresponding to the second motion sensor and recordvideo. If the camera 100 is in a low power mode, the processor 110 maydetermine whether the first camera (e.g., CAMERA 1) associated with thefirst motion sensor is on and, if so, deactivate the first camera.

In various embodiments, the processor 110 may perform video analytics onthe video being recorded to try to anticipate the motion of the movingobject being tracked. The processor 110 may be configured to control theactivation and deactivation of various image sensors in order tomaintain a seamless video recording of the motion of the object in thearea being monitored by the camera 100.

Referring to FIGS. 7A-7B, a diagram is shown illustrating anotherexample process in accordance with an example embodiment of theinvention. In an example, a method (or process) 300 may be performedusing the camera 100. The method 300 may detect motion within amonitored area and provide a video record of an object associated withthe motion detected. In an example, the method 300 may comprise a step(or state) 302, a decision step (or state) 304, a step (or state) 306, astep (or state) 308, a decision step (or state) 310, a step (or state)312, a step (or state) 314, a decision step (or state) 316, a step (orstate) 318, a step (or state) 320, a decision step (or state) 322, astep (or state) 324, a decision step (or state) 326, and a step (orstate) 328.

The process 300 may start in the state 302. In the state 302, theprocessor 110 may monitor a first motion sensor and a second motionsensor to detect motion in a monitored area. The first motion sensor maybe associated with a first image sensor having a first field of view.The second motion sensor may be associated with a second image sensorhaving a second field of view. Together, the first field of view and thesecond field of view may cover an area around a corner of a structure(e.g., a house, etc.). In an example, the first and the second fields ofview may overlap by one or more degrees. In the decision state 304, theprocessor 110 may determine whether one of the motion detectors has beentriggered. If the motion sensors have not been triggered, the process300 may move to the state 302 and continue monitoring the motionsensors. If the one of the motion sensors has been triggered, theprocess 300 may move to the state 306. In the state 306, the processor110 may turn on the image sensor associated with the motion sensor thatwas triggered and move to the state 308. In the state 308, the processor110 may determine whether the motion detected was caused by an object tobe tracked. In an example, the processor 110 may perform one or moreimage processing and/or computer vision operations or techniques (e.g.,feature extraction, object detection, object identification, etc.) todetermine whether the detected motion is associated with an object ofconcern (e.g., vehicle, person, etc.) or an object that may be ignored(e.g., small animal, bird, rain, etc.).

In the decision step 310, if the processor 110 determines the object maybe ignored, the process 300 may move to the state 312. If the processor110 determines the motion is associated with an object of concern, theprocess 300 may move to the state 314. In the state 312, the processor110 may turn off the image sensor and move to the state 302 to resumemonitoring the motion sensors. In the state 314, the processor 110 maybegin generating a sequence of video images comprising video from theimage sensor that is switched on (or activated) and track motion of theobject (e.g., using the one or more computer vision operations ortechniques). In the decision state 316, the process 300 may determinewhether the object in motion is moving from a current field of view(e.g., the field of view of the activated image sensor) to another fieldof view. If the object is not moving into another field of view, theprocess 300 may loop in the states 314 and 316. If the object is movinginto another field of view, the process 300 may move to the state 318.

In the state 318, the processor 110 may turn on the image sensorcorresponding to the field of view into which the object is moving andmove the state 320. In the state 320, the processor 110 may determinewhether the object is actually in the second field of view using the oneor more image processing and/or computer vision operations or techniques(e.g., feature extraction, object detection, object identification,etc.). In the decision state 322, if the object is not in the secondfield of view, the process 300 may return to the state 320. When theobject is confirmed to be in the second field of view, the process 300may move to the state 324. In the state 324, the processor 110 may turnoff the first image sensor and begin generating the sequence of videoimages comprising video from the second image sensor. The processor 110may also continue tracking the object. In the decision state 326, theprocessor 110 may determine whether the object has left the second fieldof view. If the object has not left the second field of view, theprocess 300 may loop through the states 324 and 326. When the object hasleft the second field of view, the process 300 may move to the state328. In the state 328, processor 110 may turn the second image sensoroff and the process 300 moves to the state 302. In an example where theobject begins moving toward the first field of view, the process 300 mayperform steps similar to the steps 314 through 328 with the first imagesensor.

In various embodiments, the processor 110 may perform video analytics onthe video being recorded to try to anticipate the motion of the movingobject being tracked. The processor 110 may be configured to control theactivation and deactivation of various image sensors in order tomaintain a seamless video recording of the motion of the object in thearea being monitored by the camera 100, while minimizing the amount ofpower utilized for extended battery life. In various embodiments, theprocessor 110 may be configured for low power operation. In an example,the processor 110 may comprise one or more dedicated hardware circuits(or engines or circuitry) implementing various image processing stepsand/or computer vision operations.

The functions and structures illustrated in the diagrams of FIGS. 1 to 7may be designed, modeled, emulated, and/or simulated using one or moreof a conventional general purpose processor, digital computer,microprocessor, microcontroller, distributed computer resources and/orsimilar computational machines, programmed according to the teachings ofthe present specification, as will be apparent to those skilled in therelevant art(s). Appropriate software, firmware, coding, routines,instructions, opcodes, microcode, and/or program modules may readily beprepared by skilled programmers based on the teachings of the presentdisclosure, as will also be apparent to those skilled in the relevantart(s). The software is generally embodied in a medium or several media,for example non-transitory storage media, and may be executed by one ormore of the processors sequentially or in parallel.

Embodiments of the present invention may also be implemented in one ormore of ASICs (application specific integrated circuits), FPGAs (fieldprogrammable gate arrays), PLDs (programmable logic devices), CPLDs(complex programmable logic device), sea-of-gates, ASSPs (applicationspecific standard products), and integrated circuits. The circuitry maybe implemented based on one or more hardware description languages.Embodiments of the present invention may be utilized in connection withflash memory, nonvolatile memory, random access memory, read-onlymemory, magnetic disks, floppy disks, optical disks such as DVDs and DVDRAM, magneto-optical disks and/or distributed storage systems.

The terms “may” and “generally” when used herein in conjunction with“is(are)” and verbs are meant to communicate the intention that thedescription is exemplary and believed to be broad enough to encompassboth the specific examples presented in the disclosure as well asalternative examples that could be derived based on the disclosure. Theterms “may” and “generally” as used herein should not be construed tonecessarily imply the desirability or possibility of omitting acorresponding element.

While the invention has been particularly shown and described withreference to embodiments thereof, it will be understood by those skilledin the art that various changes in form and details may be made withoutdeparting from the scope of the invention.

The invention claimed is:
 1. An apparatus comprising: a housing; a firstlens and first image sensor disposed within said housing and configuredto capture a first video image stream of a first field of view (FOV); asecond lens and second image sensor disposed within said housing andconfigured to capture a second video image stream of a second FOV; afirst motion sensor disposed within said housing and configured todetect motion in said first FOV; a second motion sensor disposed withinsaid housing and configured to detect motion in said second FOV; and acamera system on chip (SOC) with multi-sensor capabilities disposedwithin said housing and coupled to said first image sensor, said firstmotion sensor, said second image sensor, and said second motion sensor,wherein a processor of said camera system on chip (SOC) is configured togenerate a third video image stream in response to one or more of saidfirst video image stream and said second video image stream.
 2. Theapparatus according to claim 1, wherein said first lens and said firstimage sensor are mounted orthogonally relative to said second lens andsaid second image sensor.
 3. The apparatus according to claim 1, whereinsaid first FOV and said second FOV overlap by one or more degrees. 4.The apparatus according to claim 1, wherein said processor is furtherconfigured to track an object moving from the first or the second FOV tothe second or the first FOV.
 5. The apparatus according to claim 4,wherein said processor is further configured to switch said first andsaid second image sensors on and off as said object moves from the firstor the second FOV to the second or the first FOV.
 6. The apparatusaccording to claim 5, wherein said processor is further configured toutilize video analytics to determine when to switch said first and saidsecond image sensors on and off as said object moves from the first orthe second FOV to the second or the first FOV.
 7. The apparatusaccording to claim 6, wherein said processor is further configured toswitch said first image sensor or said second image sensor on prior tosaid object moving from the first or the second FOV to the second or thefirst FOV, and switch said first image sensor or said second imagesensor off after said object has moved from the second or the first FOVto the first or the second FOV.
 8. The apparatus according to claim 1,wherein each of said first motion sensor and said second motion sensorcomprises a passive infrared (PIR) sensor.
 9. The apparatus according toclaim 1, wherein each of said first motion sensor and said second motionsensor comprises a low resolution image sensor configured to detect atleast one of motion and a person.
 10. The apparatus according to claim1, wherein said processor is further configured to determine when toswitch said first and said second image sensors on and off based oninputs from said first motion sensor and said second motion sensor. 11.The apparatus according to claim 1, wherein said apparatus is batterypowered.
 12. The apparatus according to claim 1, wherein said processoris configured to communicate said third video image stream to amonitoring device using a wireless protocol.
 13. The apparatus accordingto claim 1, wherein said processor performs video analytics to predict adirection of movement of an object to allow said first or said secondimage sensor to be started ahead of time, to be ready as soon as theobject is in the first FOV or the second FOV.
 14. A method of monitoringan area using a camera system on chip (SOC) with multi-sensorcapabilities, said method comprising: orienting a first lens and a firstimage sensor disposed within a housing containing said camera SoC tocapture a first video image stream of a first field of view (FOV);orienting a second lens and a second image sensor disposed within saidhousing containing said camera SoC to capture a second video imagestream of a second FOV; orienting a first motion sensor disposed withinsaid housing containing said camera SoC and associated with said firstimage sensor to detect motion in said first FOV; orienting a secondmotion sensor disposed within said housing containing said camera SoCand associated with said second image sensor to detect motion in saidsecond FOV; and generating a third video image stream in response to oneor more of said first video image stream and said second video imagestream using a processor of said camera SoC coupled to said first imagesensor, said first motion sensor, said second image sensor, and saidsecond motion sensor.
 15. The method according to claim 14, wherein saidfirst lens and said first image sensor are mounted orthogonally relativeto said second lens and said second image sensor.
 16. The methodaccording to claim 14, wherein said first FOV and said second FOVoverlap by one or more degrees.
 17. The method according to claim 14,further comprising: using said processor to track an object moving fromthe first or the second FOV to the second or the first FOV; andswitching said first and said second image sensors on and off as saidobject moves from the first or the second FOV to the second or the firstFOV.
 18. The method according to claim 17, wherein said processorutilizes video analytics to determine when to switch said first and saidsecond image sensors on and off as said object moves from the first orthe second FOV to the second or the first FOV.
 19. The method accordingto claim 18, wherein said processor is further configured to switch saidfirst image sensor or said second image sensor on prior to said objectmoving from the first or the second FOV to the second or the first FOV,and switch said first image sensor or said second image sensor off aftersaid object has moved from the second or the first FOV to the first orthe second FOV.
 20. The method according to claim 14, wherein each ofsaid first motion sensor and said second motion sensor comprises atleast one of a passive infrared (PIR) sensor and a low resolution imagesensor configured to detect at least one of motion and a person.